Multiple interacting computers interfaceable through a physical manipulatory grammar

ABSTRACT

A plurality of multiple interacting computer devices for transferring data includes a first device having a processor and a first wireless communication module for transferring data, a second device having a processor and a second wireless communication module for transferring data, and a third device having a processor and a third wireless communication module for transferring data. When the first device is connected in substantially simultaneous wireless communication with the second device and the third device, specific data based on respective spatial positions of the first device, second device, and third device can be transferred.

FIELD OF THE INVENTION

The present invention relates to a method and apparatus for supporting ahand or touch operated user interface useful in conjunction withmultiple interacting devices. More particularly, the present inventionrelates to one or more computer processors connected to one ordeformable pieces for grasping, gripping, or touching mediated userinput.

BACKGROUND AND SUMMARY OF THE INVENTION

Reliably, quickly, and intuitively transmitting complex commands tosmall portable computers can be difficult. Small computer devices do notgenerally have sufficient computer processing power to respond reliablyto voice or handwritten (pen based) commands, keyboards are often absentor too small for accurate finger input, and conventional buttons are toolarge or support too limited a command instruction set. User interfacetechniques that rely on bulky external modules (full size infraredlinked keyboards, tethered data gloves, or camera based gesturalrecognition equipment, for example) are expensive, often not readilyavailable outside selected sites, and probably too awkward forwidespread use in conjunction with consumer level portable computingdevices.

User interface designers for portable computers have attempted tocompensate for some of these problems by constructing devices that relyon various spatial, positional, or environmental cues that manually orautomatically allow for activation of various modes in the device. Forexample, some laptop computers use the action of opening/closing the lidto initiate automatic bootup/powerdown of the computer without requiringany additional signal input (e.g. such as depressing a “start” button ortyping “l—o_g_o_n” on a keyboard) from a user. Alternatively, the use ofsmall portable computers that automatically switch control modesdepending on position, orientation, or room location have beeninvestigated. Buttonless manual control of a portable computer throughdeliberate user actions such as tilting the portable computer have alsobeen described.

However, all these solutions for interfacing with small portablecomputers have generally been limited in scope and functionality. Whatis needed is a user interface system suitable for even very smallportable computers (having volumetric dimensions on the order of a onecubic centimeter) that is powerful, can be intuitively operated by anordinary user with little training, and is still readily capable ofmodification or to extension by the user. The present invention meetsthese requirements by providing a manipulatory user interface thatresponds to a user twisting, folding, bending, squeezing, shaking,tilting, spinning, lifting, or otherwise physically manipulating thecomputer.

In the manipulatory user interface system of the present invention, themost basic level of manipulation is known as a “senseme”. A senseme isdefined as a single indivisible type of physical manipulation. A partiallist of categories of sensemes include material transformations such assqueezing, twisting, stretching; local spatial transformations such astranslation, rotation, orbiting; and environmental transformations basedon temperature, light level, or vibration. For example, a small portablecomputer may support a deformable piece having multiple embedded sensorsthat detect folding, twisting, or bending of the deformable piece by auser. This computer can also contain a number of accelerometers thatsense relative spatial information; gyroscopic, radio or infraredpositional sensors for determining absolute position; and variousthermal or photosensors that respectively detect temperature and lightlevel changes. Intentional or unintentional modifications detected byone or more of these sensor systems can provide the basis for a powerfuluser interface scheme.

As will be appreciated, each senseme category contains many individuallydistinguishable members. For example, consider the category of sensemeknown as a “pinch”, a structural transformation generally completed by auser squeezing the deformable piece between a forefinger and thumb. Apinch can be modified by varying its speed (quick or slow pinch),magnitude/intensity (light or hard pinch), portion of deformable piecepinched (top, bottom, or center of deformable piece pinched), or evenportion of body used to pinch (right handed pinch or left handed pinch),with each modification being distinguishable as a senseme capable ofbeing mapped onto a computer control command.

Although the wide variety of easily distinguishable sensemes would aloneprovide a powerful user interface to a computer, the present inventionfurther extends the flexibility of the senseme based user interface bysupporting computer control based on a “morpheme” input. The morpheme isa temporally synchronous (or overlapping asynchronous) tuple of one ormore sensemes. Note that a morpheme can (and often will) contain morethan one senseme. The sensemes combined into a morpheme can come eitherfrom the same category (the user pinches with a right hand while tappingwith a left hand finger), or different categories (the user pinches thedeformable piece with a right hand while modifying the spatial positionof the portable computer by tilting it forward).

Any morpheme can in turn be extended by participation in a “sentence”. Asentence is defined as a sequence of one or more temporally disjointmorphemes. The sentence level allows definition of a physicalmanipulatory grammar by appropriate choice of morpheme sequence, andcorollary rules governing, for example, use of active (verb like)morphemes, naming (noun) morphemes, or connectors. Other possiblegrammar constructs used in sentences may include those based on “home”systems. Home systems are general-purpose gestural languages, whosegrammar and syntax are not borrowed in any way from a host language.Examples of these languages are gestural languages developed by deafchildren of hearing parents who have not been exposed to American SignLanguage (ASL), and the “plains talk” of North American Indians, whichwas used as a trade language.

Accordingly, the present invention provides a method for inputtinginformation to a computer connected to a deformable piece that can bemanipulated, and optionally to various position sensors (both relativeand absolute), pressure sensors, thermal sensors, or even light sensors.The method comprises the steps of manipulating the deformable piece toprovide a first morpheme input to the computer, with the first morphemeinput normally triggering a first default action by the computer. Thedeformable piece may also be manipulated to provide a second morphemeinput to the computer, with the second morpheme input converting thenormally triggered first default action to a second action. The firstand second morphemes (and any subsequent morphemes) together form asentence that can be interpreted as a command to implement a computercontrolled action, whether it be to unlock an electronically controlleddoor, display a graphical image on a computer display, or begin loggingon to a computer network. Advantageously, such a user interface systemis well suited for interaction with small computer devices, and may evenbe cross-cultural to a limited extent, with ability to squeeze or pinchbeing universal human attributes.

The present invention is particularly useful for portable computers thatcan be held in the palm of a hand. If the portable computer is partiallyor completely surrounded with a deformable material having embedded orcontact pressure/deformation sensors, a user is able to gesture with ormanipulate the whole computer in order to achieve some desired result.Material deformation can be implemented at various scales. For instance,a computer and connected deformation sensors mounted inside a flexibletube could respond to right angle bending of the tube, or even complexknonting or looping of the tube. In practice however, only minutesurface deformations are required, being just sufficient to providetactile feedback to pressing, pinching, or bending type manipulations.In either case, however, the measurement of location and pressureapplied to a surface is sufficient to characterize the mode ofinteraction (distinguishing a pinch from a prod).

One particularly preferred embodiment of a handheld portable computerthat responds to a physical manipulatory grammar in accordance with thepresent invention includes a computer, a feedback module to providevisual, auditory, or tactile feedback to a user (e.g., processorconnected LCD display, audio speaker, or tactile display to presentBraille or other conventional touch interface), and co-mountedgraspable, deformable piece partially or completely surrounding thefeedback module. In addition, various thermal or pressure sensors aremounted to detect handedness of a user, with the grasping hand generallybeing the non-dominant hand for the user. Depending on the handedness ofthe user (which can be considered as a morpheme), the displayed datastructure is modified. For example, text displayed on an LCD displayscreen may be automatically shifted rightward on the screen to allow penbased annotation on the left side of the screen, thereby aiding lefthanded users.

Physically manipulatable user interfaces additionally provide anopportunity for multiple devices to interact in a user friendly manner.For example, a tileable display system having multiple displays withtouch sensitive deformable pieces surrounding the display can be used toorganize documents based on relative position of contact of eachdisplay. For example, if two displays initially showing separate datastructures (e.g two different pages from two different electronic books)are brought together in side by side touching contact, the displayedvisual information can alter (e.g. by displaying adjacent pages of asingle electronic book). As will be appreciated, one can also usemultiple computers connected to physically manipulatable controlelements to construct complex commands for organizing data structures.

Additional functions, objects, advantages, and features of the presentinvention will become apparent from consideration of the followingdescription and drawings of preferred embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an approximately spherical handholdable portable computer having a deformable surface, a statusdisplay, and a pressure sensor array for detecting surface deformations;

FIG. 2 is a graphical diagram illustrating various physical manipulationmorphemes, with axes respectively illustrating complexity of sensemetuples required to form a morpheme, and plasticity of a device requiredto support the physical manipulation;

FIGS. 3-16 schematically illustrate various preferred classes ofphysical manipulation morphemes;

FIG. 17 is a graphical diagram illustrating various spatial morphemes,with a first axis illustrating complexity of senseme tuples required toform a morpheme, and a second axis illustrating the degree of positionalinformation needed to support the physical manipulation (moving along acontinuum from relative local measurement along a single specifieddimension to absolute global measurement with six degrees of freedomdetermined);

FIGS. 18-26 schematically illustrate various preferred spatialmanipulation morphemes;

FIG. 27 is a graphical illustration showing increasing degrees of sensorsystem complexity that can be used to detect various categories ofenvironmental stimuli, including light effects, thermal effects, theelectromagnetic environment, and the vibratory/acoustic environment;

FIG. 28 is a graphical diagram illustrating various physicalmanipulation morphemes for multiple interacting devices, with axesrespectively illustrating the complexity of senseme tuples required toform a morpheme, and increasing level of physical contact;

FIGS. 29-34 schematically illustrate various preferred manipulationmorphemes for multiple interacting devices;

FIGS. 35 and 36 are schematic diagrams illustrating “squeeze” and “tilt”morphemes applicable to a portable computer;

FIG. 37 is a schematic diagram illustrating tilt and squeeze morphemesused to control view of large two dimensional data sets with arelatively small display of a portable computer;

FIG. 38 is a schematic illustration representing a portable computerhaving a display ready to receive annotations from a right handed user;

FIG. 39 is a schematic illustration representing a portable computerhaving a display ready to receive annotations from a left handed user;

FIG. 40 is an electronic schematic illustrating components of thepressure and tilt sensitive modules of a portable computer such asillustrated in FIGS. 35-39.

FIGS. 41 and 42 are schematic illustrations of a scanner/printer/copierusing a paper shaped display interface to support morphemic input;

FIGS. 43-45 are schematic illustrations of tilable displays capable ofsupporting morphemic input;

FIG. 46 illustrates optical sensors and patterns suitable for use inconjunction with tileable displays such as illustrated in FIGS. 43-45;

FIG. 47 illustrates radio transponders suitable for use in conjunctionwith tileable displays such as illustrated in FIGS. 43-45; and

FIG. 48 illustrates addressing of multiple tileable displays.

DETAILED DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an embodiment of the present invention suitable forsupporting a morphemic user interface grammar. Support of the grammarcan require detection of a user's physical manipulation of a device,detection of relative or absolute spatial location of the device,detection of various environmental factors acting on the device, andeven detection and interaction with multiple devices or externalcomputer networks. As illustrated, a device 10 has a deformable surface20 with an underlying deformation sensor mesh 22 for detecting surfacedeformation across or within multiple subregions of the deformablesurface 20. The deformation sensor mesh 22 is connected to an internallycontained processor 24 having associated memory system 26. For detectingvarious positional or environmental variables, a sensing system 28 isalso provided. The illustrated device further includes a feedback module33, which may include an externally visible status display 30 or anon-visual feedback module 31 (typically delivering auditory or tactilefeedback). In the illustrated device, a communications system 32 forreception or transmission of information to other electronic orcomputing devices is also provided. All these components can be poweredby a power supply 25, which is usually an internally mountedrechargeable battery of conventional construction.

Although the device 10 is illustrated as having an approximatelyspheroidal and unitary mass, various other shapes are contemplated to bewithin the scope of the present invention. For example, the overallshape may be similar to various rectangular prisms, or can beellipsoidal, toroidal, planar, or even be malleable enough to support awide range of user defined irregular shapes. In addition, multiplecooperating shape elements are contemplated using conventional designsthat permit interlocking of multiple shape elements (e.g using a balland socket, a lock and key, or slidable or rotatable interlockedcomponents).

Whatever the shape of device 10, for operation of the present inventionthe device 10 is completely or partially enveloped by the deformablesurface 20. The present invention supports use of a great variety ofdesigns and materials for the deformable surface 20, depending on therequired plasticity, durability, longevity, and of course, costconstraints. For example, contemplated designs for deformable surface 20include, but are not limited to:

a closed or open celled polymeric foam material having a wall thicknessof millimeters to centimeters, with thinner walled embodiments beingsupported (e.g. by adhesive attachment) by an internal hard shell(constructed from polymeric or metallic materials), and those thickerwalled embodiments directly supporting (by, e.g. brackets or supports)internal components such as processor 24. Suitable foams may includethose composed in whole or in part of widely available synthetic rubberssuch as polychloroprene (neoprene), polystyrenes, rubber or nitrilerubber latex foams, polysiloxanes, block polymers includingstyrene-butadiene or styrene isoprene, or any other conventionalmaterial having good elasticity and deformability;

a thin single layer polymeric surface loosely wrapped around a internalhard shell (the hard shell being constructed from polymeric or metallicmaterials). For example, a nylon or cotton weave, single layerpolyethylene, synthetic rubber (with little or no foam cells present),or natural polymeric materials such as leather wrapped around apolystyrene casing can be used;

a composite layered surface having a durable polymeric outer layersupported by an inner foam layer; or even

a polymeric bilayer having an intermediate fluid or gel layer of aviscous or thixotropic material that can be used to support extremedeformations. The intermediate layer can be relatively thick (one theorder of centimeters), or in certain embodiments can have a thicknessmeasured on micron to millimeter scales. Such extremely thin layerswould allow complex twisting, folding, curling, or crumpling actions,and have been described in conjunction with U.S. Pat. No. 5,389,945,assigned to Xerox Corp., the disclosure of which is herein specificallyincorporated by reference.

The deformation sensor mesh 22 can be embedded within, or positioned tocontact, the deformable surface 20. The deformation sensor mesh 22 caninclude an array of individual compressional or tensional strainsensors, or alternatively, embedded or attached positional sensors. Forcertain applications, continuous sensors (e.g. bilayer sheets ofcapacitance sensors) may be employed. One particularly useful continuoussensor type uses multiple capacitance or resistance strips, withdeformation pressure resulting in a positionally localizable analogsignal proportional to the applied deformation pressure. Various sensortypes can be used, including simple capacitance sensors, resistivestrain sensors, analog or digital pressure switches, inductive sensors,or even fluid flow sensors. Depending on the sensor type employed,sensor data can be directly fed to the processor 24 in digital form, orbe transformed to digital format by an general purpose analog/digitalconverter that typically provides a 4 or 8 bit range (although as few asone or as many as 32 bits may be required by various applications). Ananalog to digital converter may be internal to the processor 24 orprovided as an external module. As will be appreciated, the sensor mesh22 is intended to include combinations of sensors and sensor types,which can be used over the whole or part of the deformable surface 20.

A positional or environmental sensor system 28 can also be supported bydevice 10. Various sensor modes can be supported, including absolute orrelative positional information as determined by gyroscopic sensors,accelerometers, or acoustic or infrared ranging techniques.Environmental sensors, including conventional light, image, thermal,electromagnetic, vibratory, or acoustic sensors can also be present.Depending on the desired application, even costly environmental orpositional sensors such as those incorporating differential GPSpositioning, image analysis or recognition, acoustic or voiceidentification, or differential thermal sensors can be used as morphemicinput. Such morphemic input, taken in conjunction with morphemic inputas detected by sensor mesh 22, can enhance precision and flexibility ofa user's control of device 10.

As illustrated, both sensor system 28 and sensor mesh 22 are connectedto the processor 24 and associated memory 26. The processor 24 andmemory 26 are typically mounted within the deformable surface 20, byeither direct attachment to the deformable surface 20 or by attachmentto a hard casing positioned within the deformable surface 20.Conventional CISC or RISC processors can be used in the illustratedembodiment, with low power processors such as the Signetics 87c752 or87c751, Motorola 68HC11 or 68582, or ARM 710 being preferred. Ifconvenient, coprocessors such as analog to digital converters or digitalsignal processors can be used alone or in conjunction with a mainprocessor. Conventional flash, static, or dynamic RAM can used in thepresent invention, although for certain applications higher costembedded DRAM may also be used. In some storage intensive applications,memory 26 can include additional harddisk storage, either located withinthe device 10 or available through an external connection. As will beappreciated, for many applications use of optional externalcommunications can at least partially supplant use of internalprocessors and memory (except for that necessary to support requiredsensor or communication buffering and signalling).

The present invention optionally supports communications with anexternal computer system 40 using its internal communications system 32and associated transceiver 34. The external computer system 40 alsoincludes a transceiver 42, a personal computer or workstation 44, and isconnected to a local or wide area network computer system 46. Thetransceivers 34 and 42 can support various communication protocols anddesigns, including use of a serial tethered line 36 (using, for examplethe RS-232C interface protocols), use of infrared signals 38 adhering towidely utilized IRDA communication standards, or use of radiofrequencysignal 37 (which can be, for example, a cellular telephone, 900 MHzradio, or digital PCS telephonic communications). Alternativecommunication standards, or even alternative communication carriers suchas those based on optical or acoustic techniques, can of course beemployed.

As will be appreciated, in addition to direct communication withexternal computer system 40, the device 10 can be directly or indirectlymaintained in continuous or intermittent communication with a number ofsuitably equipped electronic devices, including a tablet computer 110,or even a physically manipulatable portable computer 11 similar indesign and function to device 10. Communication can be direct to atarget device, or through an intermediary retransmitter such as computersystem 40. Other possible communication targets include automationcontrol systems, security authorization units, personal digitalassistants, notebook computers, or any other suitably equippedelectronic system.

Results of communications with external devices, presentation of devicestored information, or device status updates can all be provided to auser through processor 24 controlled update of feedback module 33.Feedback to a user can be primarily visual, such as can occur inconjunction with visual display 30. Generally, the display 30 can be aconventional passive or active matrix liquid crystal display, althoughuse of more sophisticated (and expensive) displays based on variouselectrooptical or micromechanical techniques can of course be used. Inaddition, for certain devices a non-imaging display such as may beformed by a small number of status lights (e.g. red or green LEDs), orlocalized or distributed chromatic changes (in conjunction with adeformable surface 22 constructed with suitable electrochromicmaterials) may be all that is necessary for visual feedback to the user.

In some embodiments of the invention, visual output through display 30may be augmented (or even replaced) with a non-visual display 31. Thenon-visual display 31 can include tactile displays based on internalactuators, auditory feedback, or even displays based on conformalchanges in device appearance. For example, one possible feedback displayis based on internal auditory speakers (emitting a range of sounds fromsimple “beeps” to well formed speech, depending on available processorspeed and functionality) for providing user feedback. As will beappreciated, non-visual display 31 and its associated actuators orelectronics can support alternative feedback modes, including, forexample, force feedback to a user through internal actuators, tactilebased feedback (e.g with multiple surface projections for presentationof Braille or other conventional tactile user interface), modificationsto the surface texture of the device, or any other conventionalmechanism for supplying information to a user.

To better appreciate operation of the present invention, some selectedmodes of physical manipulation of device 10 are schematically presentedin FIG. 1. As seen in FIG. 1, device 10 can be translationally moved inthree spatial dimensions, as illustrated with reference to orthogonalforce arrows 50, 51, and 52. In addition to translational motion, thedevice 10 can be rotationally moved in any or all three spatialdimensions, as represented by arrows 53, 54, and 55. By use of sensorsystem 28 (alone or in combination with communication system 32), therelative or absolute position and orientation in three dimensions can bedetermined.

In addition to determining spatial position and orientation through useof sensors 28, the device 10 can optionally use sensor mesh 22 tomeasure and localize transient or continuing force application, withforce vectors and related timing information being determined andinterpreted. A few possible force actions (deformation modes) areschematically illustrated in FIG. 1, with arrows 60 and 61 illustratingdepression of surface 20 (with the combination representing a squeeze),arrows 62 and 63 illustrating sliding or rubbing deformation (with thecombination representing a twist), and the combination of sliding arrows65 and 66, and outward pull 67, together representing a pinch andoutward pull. The strength of the applied force can be measured (e.g. ahard or soft squeeze is differentiated), its spatial expanse found (e.g.to differentiate between poking with a fingertip or a thumb pad), andtiming determined (e.g. to differentiate between a quick or a slowdepression of the surface). The deformation so caused may be eitherpermanent or transitory.

As will be appreciated by those skilled in the art, each of theforegoing force actions represented by arrows can be considered asenseme. Some temporally distinguishable sensemes (or combinations ofsensemes such as the foregoing discussed pinch/pull combination) furtherrepresent morphemes used as a basis for a morphemic grammar inaccordance with the present invention. All of the following describedmorphemes can be modified by numerous variations in applied pressure,force utilized, appendages used, body parts, or extraneous intermediaryobjects used to apply force. In addition, timing of various objects(whether quick, slow, or alternately quick and slow) can modifyinterpretation of a morpheme. For example, if “squeeze” is taken as atypical morpheme, one can appreciate various squeeze operations such asquick squeeze, slow squeeze, hard squeeze, soft squeeze, narrow squeeze,wide squeeze, squeeze between hands, squeeze between one hand and auser's chest or head, squeeze between one hand and a table or wall, asqueeze made between two pens or two books, or even a squeeze between auser's tongue and the roof of the mouth. For purposes of the presentinvention, all such squeeze morphemes would be considered members of the“squeeze” class, with individual variations acting as possible modifiersor selected cases, just as a “house cat” might be considered aparticular member of the class of “felines”, which more generallyembraces lions, tigers, and bobcats.

To aid in understanding the diversity of contemplated physicalmanipulation morphemes, FIG. 2 illustrates selected morphemes arrangedby increasing plasticity of the device required to enable morphemeutilization, and by increasing complexity of available senseme tuplesrequired to form or interpret a morpheme applied to a particular classof device. Beginning with the least plastic device and the simplestsenseme set used to compose a morpheme, a definition of a possiblephysical manipulation and typical function invoked by that manipulationof a device similar (but of course possibly more complex) to thatdescribed in conjunction with FIG. 1 is presented:

DEPRESS

Definition: Indenting one or more subregions of the device through theapplication of pressure.

Example: As seen in FIG. 3, consider a device 122 having a display 123.The device 122 supports a graphical illustration software applicationwhich allows users to position geometric shapes or user definedgraphical objects. The device can have four pads 124 placed around it,one on each side. By deforming a particular side, the user indicates adesire to “nudge” the currently selected geometric shape 125 away fromthat side to a new position 126.

SQUEEZE

Definition: To deform one or more subregions of a device through theapplication of vectors of force, some components of which are directedtowards each other, which compress the morphological structure of thedevice. This is a special case of depression.

Example: As illustrated in FIG. 4, consider a device 132 which candisplay one or more documents in either iconic (closed) or textualdisplay (open) modes. By selecting an open document 135, and thensqueezing a deformable edge 134 of the device 132, the user indicates adesire to make the document “smaller”, in this case to iconize it asicon 136.

FOLD

Definition: To deform a second subregion by bending the first subregionsuch that it partially or completely overlaps the second subregion.Further deformations can be applied to that new morphological structureon other subregions.

Example: As illustrated in FIG. 5, consider a device 142 which candisplay documents. Suppose this device 142 has been augmented such thata deformable horizontal “flap” 144 on a top edge of the device 142 canbe folded to partially obscure a display 143. When the user makes thisfolding gesture, the user indicates a desire to password-protect(“hide”) the currently displayed document.

CURL

Definition: Deforming one or more subregions of the device by spirallyconfiguring subregions relative to each other in a cylindrical orcircular morphological structure.

Example: As illustrated in FIG. 6, consider a device 150 which candisplay documents in various languages (English, French, etc.). When theuser takes such a device 150, curls in a direction indicated by arrow157 into a tube, and then uncurls it, this “abracadabra” gesture tellsthe device 150 to display the current document in a different language.

STRETCH

Definition: Deforming one or more subregions of a device through theapplication of vectors of forces, some components of which are directedaway from each other, the vectors being applied at opposite ends of thedevice. Example: As illustrated in FIG. 7, consider a device 160 with agraphical software application which allows users to manipulategeometric shapes. By stretching the device 160, the user indicates adesire to “resize” or “rescale” the currently displayed shape 165 to alarger size 166, the amount of resizing being a function of the amountof deformation. Note that SQUEEZING can indicate resizing the currentlydisplayed to a smaller size.

PINCH

Definition: Manipulating one or more subregions by applying vectors offorces, aligned directly towards each other, on opposite sides of theaffected subregion(s). This is typically, but not exclusively,accomplished using two-finger tactile force. PINCH is a special case ofSQUEEZE.

Example: As illustrated in FIG. 8, consider a device 170 which can copydocuments. By performing a “pinching” action 175, the user indicatesthat they wish the next set of copies to be issued in stapled form.

DOGEAR

Definition: Deforming a second subregion by folding a first subregion ata logical corner or edge of the second subregion, indicating a marker orlocation point (e.g. a bookmark) to be later referenced. DOGEAR is aspecial case of FOLD.

Example: As illustrated in FIG. 9, consider a device 180 which displaysa subset of the pages from a multi-page document. By “dogearing” theupper right corner 185 of device 180, the user indicates that they wisha bookmark associated with the currently displayed page or pages.

TWIST

Definition: Deforming one or more subregions of the device through theapplication of two opposing rotational forces offset from each other bya non-zero difference about some central axis.

Example: As illustrated in FIG. 10, consider a device 190 which, overtime, becomes degraded in some aspect of its performance (its diskbecomes fragmented, its memory needs garbage-collection, etc.). Byperforming a “TWIST” gesture 195, the user indicates that they wish thedevice to “wring itself out”, performing, for example, garbagecollection.

RELIEF-MAP

Definition: Deforming one or more subregions of the device by raisingand or lowering them by either spatial transformation or theaddition/removal of material.

Example: As illustrated in FIG. 11, consider a device 200 which candisplay documents in either one- or two- page format. When the user“scores” the device by making a vertical indentation about the centeraxis of the device while it displays a single page 206, the device 200interprets the morpheme to request display of documents in a two-pageformat as pages 207 and 208.

RIP

Definition: Deforming one or more subregions of the device byintroducing a spatial discontinuity, by applying vectors of force topartially or totally disconnect these subregions from the device.

Example: As illustrated in FIG. 12, consider a device 210 which can copysome or all of its information. When the user performs the “ripping”gesture 215, removing one or more subregions, the device 210 copies itscurrently selected data set onto those subregions 216 and 217.

PERFORATE

Definition: Deforming one or more subregions of the device by means ofintroducing a change in the spatial connectivity of the subregions suchthat a hole is introduced (either temporarily or permanently) in thedevice.

Example: As illustrated in FIG. 13, consider a device 220 which is usedto route messages between various parts of a device network, and whichdisplays this functionality to the user by means of lines 222representing message pathways. When the user perforates the device 220with a finger or object 224 to introduce a hole in one of these pathways222, the system stops routing messages along that pathway.

SIMILARITY

Definition: Deformation of one or more subregions of a device previouslyconfigured to represent some other predefined object. Typically thedevice acts in a manner consistent with the behavior of the real-worldobject when it is manipulated in this manner.

Example: As illustrated in FIG. 14, consider a device 230 which containstextto-speech and audio input capability, and which presents itself tothe user in an anatomically accurate shape of a human head. When theuser opens lips 232 on the human head, internal sensors detect openingof the lips and activate text-to-speech capability.

3D MAP

Definition: The morphing of a device that can be molded around anexternal object to permit a sensor mesh to determine simultaneously sizeand shape of the external object. The range of possible external objectsis large but is limited by the size of the device's solid inner-housingand the volume of the moldable material in the outer housing. In thissystem the device has the ability to accurately sense the amount ofmaterial from its inner surface to the outer edge of the moldablematerial (example: via ultrasonic sounding, similar to sonar in water)thus determining an accurate electronic model for the shape of themolded enclosure.

Example: As illustrated in FIG. 15, by pressing a device attachedmoldable material 244 around the surface of an external object (e.g. acog 242) a device 240 can automatically generate a CAD model of thatobject and store it in its memory.

MIMICRY

Definition: Deforming one or more subregions of the device such that theresultant morphological structure resembles a known real-world objectand through this association of subregions, the device acts in a mannerconsistent with the object it resembles.

Example: As illustrated in FIG. 16, consider a device 250 containing acomputer which contains text-to-speech and audio input capability, andwhich presents itself to the user as a misshapen blob having theconsistency and plasticity of moldable putty or clay. When the userperforms the “MIMICRY” action by molding part of the device 250 toresemble an ear, the audio input capability is activated.

In addition to morphemes based on physical manipulation, variousmorphemes based on varying degrees of relative or absolute spatialpositioning are contemplated to be useful in practice of the presentinvention. To aid in understanding the diversity of contemplated spatialmorphemes, FIG. 17 illustrates selected spatial morphemes arranged byincreasing knowledge of spatial position required to enable morphemeutilization, and by increasing complexity of available senseme tuplesrequired to form or interpret a morpheme applied to a particular classof device. Beginning with a device having only rudimentary relativepositioning functionality for supporting simple spatial sensemes, andending with a device absolutely positionable to within centimetersanywhere on Earth, a definition of a possible spatial manipulation andtypical function invoked by that manipulation of a device similar (butof course possibly more complex) to that described in conjunction withFIG. 1 is presented:

TRANSLATE (relative to device)

Definition: The linear movement of a device's center of mass from oneposition in space to another.

Example: Used to substitute for mouse controlled graphical “sliders” inconventional graphical user interfaces. When it is only physicallypossible to display a small amount of list, large listings can still besearched by “scrolling” a display window in response to the TRANSLATEmorpheme.

SHAKE

Definition: Spatially translating all subregions of a device by repeatedmovement in opposing directions, such that the net translation isnegligible.

Example: As illustrated in FIG. 18, consider a device 260 which is usedas a calculating device. When the user performs the “SHAKE” gesture, thedevice 260 clears its accumulator.

REVOLVE

Definition: Rotating all subregions of a device by rotating thesubregions about a point internal to the device, about any arbitraryplane.

Example: As illustrated in FIG. 19, consider a device 270 which displaysan imaged slice of volumetric data, such as medical data from a seriesof CAT scans. By rotating the device about a center point 272 internalto the device 270 to a new position 274, the plane specifying the imagedslice is changed accordingly.

TILT

Definition: Rotating one or more subregions of a device by rotating thesubregions such that one or more components of the rotary force are inthe direction of gravity and the amount of rotation is between about−180 degrees and +180 degrees.

Example: Consider a device which displays frames from an animationsequence on the side facing the user. As the device is tilted away, thespeed of the animation increases—as its tilted towards the user, thespeed of the animation decreases, analogous to operation of a gas pedal.

FLICK

Definition: A forwards TILT immediately followed by an opposingbackwards TILT.

Example: As illustrated in FIG. 20, consider a device 280 which cantransmit some subset of its data to another device. When the userperforms the “FLICK” gesture by quickly tilting in the direction ofarrow 282, followed by a reverse tilt along arrow 282, the device 280performs this transmission, towards the device (not shown) pointed to bythe ray of the gesture.

SPIN

Definition: Rotating one or more subregions of the device by rotatingthe subregions about a point internal to the device, such that the planeof rotation is one of the device's surface planes. SPIN can beconsidered a special case of REVOLVE.

Example: As illustrated in FIG. 21, consider a device 290 which candisplay a frame of video 295 from a video sequence. The user performsthe “SPIN” gesture in a counter-clockwise direction, the device displaysearlier frames in the sequence; when the gesture is performed in aclockwise direction (arrow 292), the device 290 displays a later frame296 in the sequence (represented by film strip 294).

ORIENT

Definition: Rotating one or more subregions of the device by rotatingthe subregions about the center of the device, such that the plane ofrotation is one of the device's surface planes, and the amount of therotation is a multiple of 90 degrees (i.e. to rotate the device betweencardinal compass points). Orient can be considered a special case ofSPIN, which in turn is a special case of REVOLVE.

Example: As illustrated in FIG. 22, consider a device 300 which candisplay a document in either 1 page, 2-page, or 4-page format (“1-up”,“2-up”, or “4-up”). When the user performs the orient gesture in aclockwise direction (arrow 302), the device 300 increases the number ofdocument pages it is displaying from one page 305 to two pages 306 and307. Further orient gestures would increase the number of displayedpages. When performed in a counter-clockwise direction, the device 300decreases the number of pages it is displaying.

FACE

Definition: Manipulating one or more subregions of the device such thata first set of subregions is no longer bottom-most and a second distinctset of subregions now assumes the first subregions former position.

Example: As illustrated in FIG. 23, consider a device 310 which displaysdocuments, and which allows users to edit such documents. Considerfurther the case in which the device presents itself to the user in theform of a cube, in which 6 different documents are displayed on the 6different faces. When the user performs the “FACE” gesture by making aparticular face top-most, the document which is now on the top-most facebecomes editable by the user, while the document which is no longertop-most is no longer editable.

LIFT

Definition: The movement of a device's center of mass in a directionopposite to the current gravitation force acting on the device.

Example: Commanding the device to display the computer's file system ata position one level higher in the hierarchy.

PAN

Definition: The application of a TRANSLATION to a device such that it ismoved parallel to the front of the user's body, at a substantiallyconstant height.

Example: Viewing a spread-sheet on a device with a display so small onlyone cell can be shown. By PANing the device, the contents of the currentrow can be shown in sequence depending on the rate or amount of PAN.However, if the device were rotated away from its current orientationduring the PAN, a new row would be chosen. The selection of the rowcould be dependent on the deviation from the original orientation.

PUSH-PULL

Definition: Manipulating one or more subregions of the device byspatially translating them such that they are moved along a line ofprojection from the center of the device to the vertical axis of theuser's body.

Example: Consider a device with audio output capability. As the deviceis “pushed” further away from the body, its audio output levelincreases. When it's “pulled” towards the body, its level decreases.

WHACK

Definition: The application of an accelerative or de-accelerative forceto one or more subregions of a device such that said subregions contactor are contacted by an external object, causing an equal and oppositecountering force.

Example: As illustrated in FIG. 24, consider a device 320 which canperform long and unpredictable database searches. When the user performsthe WHACK gesture (e.g. upon a table 322), the current search isaborted.

ORIENT (relative to environment)

Definition: Manipulating two subregions of the device such that the linedrawn between the centers of those two subregions alters its orientationwith respect to the surrounding environment.

Example: Displaying a CAD drawing of a machine-part in 3D on the displayof a mobile device. As the orientation of the device changes, so toodoes the viewing angle and position of the rendered image.

ORBIT

Definition: Rotating one or more subregions by rotating the subregionsand/or the center of mass of the device about some point exterior to thephysical boundaries of the device, about any arbitrary axis.

Example: As illustrated in FIG. 25, consider a device 330 which cansearch a network database, such as the World Wide Web, for information.When the user performs the “orbit” gesture, such a search is initiated.The radius 332 of the rotation 334 specifies the breadth of thesearch—wider circles specify a wider search. The speed of the gesturespecifies the time limit imposed on the search—the quicker the gesture,the more cursory the search.

ORBIT RELATIVE TO USER

Definition: Rotating one or more subregions by rotating the subregionsand/or the center of mass of the device about some point exterior to thephysical boundaries of the device, where said point is proximal to abody feature of the user. This is a special case of ORBIT.

Example: As illustrated in FIG. 26, consider a device 340 which canperform audio output. By performing the ORBIT gesture (in directionindicated by arrow 344) about the user's ear 345, the audio output isactivated.

MOVEMENT IN ROOM

Definition: The local detection of a device's 3D position relative toreference points found within an enclosing room. Differences in themeasured position are used to trigger actions.

Example: A virtual filing system that allows you to save and restorefiles based on the devices current position in the room. To save a fileyou might think carefully about the contents of the file and then walkto the position in the room that might be most easily associated withit. When retrieving the file you would use the same thought processesand go back to the position you had associated with the file. On doingso the files associated with that position would be displayed and youwould then be able to select the file you were looking for. The systemis useful because the human mind is very good at remembering informationthat is spatially organized rather than in some abstract informationdata structure.

MOVEMENT BETWEEN WIDELY SEPARATED SITES

Definition: Manipulating one or subregions of the device such that thedetected absolute spatial position of those subregions is changed.

Example: Consider a device which can display information from a databaseof client information. When the device is moved to a different clientsite, the device automatically updates its display to displayinformation for the nearest client site.

In addition to morphemes based on physical manipulation or spatialpositioning, various morphemes based on sensed environmental conditionsare contemplated to be useful in practice of the present invention. Toaid in understanding the diversity of contemplated environmentalmorphemes, FIG. 27 illustrates selected environmental morphemes looselyarranged in order of increasing sensor complexity needed in somecommonly sensed environmental categories. For each of the profferedcategories, some selected sensing systems supportable by a device suchas that described in conjunction with FIG. 1 are presented:

LIGHT

Definition: Manipulating one or more subregions of the device such thatthe amount of light falling upon those subregions changes.

Example: Consider a device used in a lecture hall to take notes. Whenthe room lights are turned on, the light sensors detect this and turndown the backlight to conserve energy. When the room lights are turnedoff (e.g. during a slide show) the light sensors detect this and turn upthe backlight to increase viewability.

Light sensors can range from thresholded binary light detectors, tolight pattern detectors, to full imaging systems. Advanced techniquescan include image analysis and recognition to identify objects orpersons.

HEAT

Definition: Manipulating one or more subregions of the device such thatthe amount of heat applied to those subregions changes.

Example: Consider a portable computer which has a stylus for enteringtext. By looking at the heat profile along the back surface of thecomputer, the computer can detect whether it is being held with the lefthand, the right hand, both hands, or neither hand, and update itsinterface accordingly.

Thermal (heat) sensors can range from simple temperature sensors tosophisticated differential thermal mappers and thermal imagers.

ELECTROMAGNETIC

Definition: Manipulating one or more subregions of the device such thatthe electromagnetic spectrum applied to those subregions changes.

Example: By analyzing the radio spectrum, the device can deriveestimates as to its absolute spatial position, and use that to alter itsfunctionality.

Electromagnetic detection can include magnetic compasses, radiodetection, or GPS signal detection. More advanced techniques can includeelectromagnetic spectrum analysis and interpretation, such as roughlydetermining location based on available radio signals.

VIBRATE

Definition: Manipulating one or more subregions of the device byvibration.

Example: Consider a device which displays textual information. When theuser takes the device on the bus, the ambient vibration level sensed bythe device changes, and the device increases the size of the displayedtext to help the user compensate.

This class of environmental morphemes can include detection ofintermittent contacts, low frequency rumblings, or acoustic leveldetection. More advanced techniques requiring greater processor powerinclude maximum frequency identification, spectral analysis of acousticfrequencies (enabling the device to distinguish background environmentalnoises from speech, for example), or even speech based identification ofpersons in the vicinity of a device.

In addition to morphemes based on physical manipulation, spatialposition, or sensed environmental factors, various morphemes based oncooperation between multiple interacting devices are contemplated to beuseful in practice of the present invention. To aid in understanding thediversity of contemplated spatial morphemes, FIG. 28 illustratesmultidevice morphemes arranged by an increasing level of possiblephysical contact, and by increasing complexity of available sensemetuples required to form or interpret a morpheme applied to a particularclass of device. Beginning with a device having only rudimentary edgedeformation functionality for supporting simple spatial sensemes, andending with complex deformable or embeddable devices that can be wrappedabout each other, a definition of a possible multidevice manipulationand typical function invoked by that multidevice manipulation of devicessimilar (but of course possibly more complex) to that described inconjunction with FIG. 1 is presented:

TOUCH

Definition: To move one or more subregions of a device such that theyenter physical contact with a subregion of a second device, in anyalignment and to any extent. Or, to take two devices so aligned andremove that alignment.

Example: Consider two portable computers 350 and 351, the first of whichcontains a database, and the second of which contains an IRDA port. Whenthe user touches the first computer 350 to the second computer 351, thedatabase is transmitted via the second computer's port.

MATCH

Definition: To move one or more subregions of a device such that theyenter physical contact with one or more subregions of a second device,with the subregion(s) of the first device and the subregion(s) of thesecond device being aligned along one or more edges. Or, to take twodevices so aligned and remove that alignment.

Example: As illustrated in FIG. 30, consider, multiple devices 360, 361,and 362, which contain different versions of the same basic database.When the user MATCHes the first device 360 to the second device 361,followed by matching the third device 362 to the matched first andsecond devices, their databases are reconciled (synchronized).

STACK

Definition: To move one or more subregions of a device such that theyenter physical contact with a subregion of a second device, such thatthe first device is now located above, but physically adjacent to, thesecond device. Or, to take two devices so aligned and remove (i.e.unstack) that alignment.

Example: As illustrated in FIG. 31, consider a set of devices 370, 371,and 372, each of which is displaying a frame of video from a longervideo sequence. When the devices are stacked, the ordering of thestacking specifies an order for the video editing, and a singlecomposite video is now produced.

TILE

Definition: To move one or more subregions of a device such that theyphysically contact with a subregion of a second device, such that thefirst device and second device now form a single seamless spatial unitor to take two devices so aligned and remove that alignment. TILE is aspecial case of MATCH.

Example: As illustrated in FIG. 32, consider a set of devices 380, 381,382, is 383, each of which can independently display a portion of alarge photograph. When the devices are tiled, each device displays theportion of the photograph appropriate to its current relative positionin the tiled grid.

RELATIVELY ALIGN

Definition: To move one or more subregions of a device such that theyengage in a particular spatial relation to one or more other devices,where said devices are not touching.

Example: As illustrated in FIG. 33, consider a set of devices 390, 391,392, and 393 which are displaying a multi-page document. Whicheverdevice is presently placed at the far left (device 390) displays thetable of contents, whichever one is presently placed at the far right(device 393) displays the index, and the others display pages accordingto their respective locations. As different devices can have differentdisplay capabilities, moving them about can alter the document display.For example, if only one of the devices has a color display, when it ismoved from second position to third position then (a) the device whichwas in third position, displaying page #2, now displays page #1, and (b)the color display, which was displaying page #1, now displays page #2 incolor.

WRAPPING/EMBEDDING

Definition: Manipulating one or more subregions of a device such thatthese subregions spatially occlude or are spatially occluded by someportion of the second device.

Example: As illustrated in FIG. 34, consider a first device 400 whichcontains the infrastructure to support the filtering of email. Considera second set of devices 401 and 402 which implement particular emailfilters. When the user physically embeds device 401 (or 402) into thefirst device 400, by which the first device 400 now wraps the seconddevice 401 (or 402), the particular email filter supported by the seconddevice is activated.

As those skilled in the art will appreciate, combinations of any of theforegoing described morphemes based on physical manipulation, spatialposition, environmental conditions, or multiple interacting devices canbe extended by participation in a morphemic “sentence”. A sentence isdefined as a sequence of one or more temporally disjoint morphemes.Typically, between about {fraction (1/10)}th of a second and 2-3 secondssuffices to distinguish morphemes within a sentence. Of course, in somecircumstances and indefinite time period may elapse. The sentence levelallows definition of a physical manipulatory grammar by appropriatechoice of a morpheme sequence, and corollary rules governing, forexample, use of active (verb like) morphemes, naming (noun) morphemes,or connectors. Just as the position and relation of words in a sentencedefine the sentence's meaning (e.g. “horse chestnut” is not the same as“chestnut horse”), similarly the position and relation of morphemes in amanipulatory sentence define the sentence's meaning. For example, in acommunication mode a FLICK followed by a WHACK could mean “transfer dataand erase the local copy”, while a WHACK followed by a FLICK means“power on the device and transfer data”. In other settings, a FLICK or aWHACK could mean something entirely different. To better understandconstruction of morphemic sentences, the following examples arediscussed:

DATA TRANSFER SENTENCE

Consider a device which can transmit some or all of its information toanother device. Furthermore, this transmission can be done eitherunencrypted, or encrypted (to increase security). Furthermore, thistransmission of a text+graphics document can either include thegraphics, or omit them (to save time). Suppose that the user wishes toperform the command “Transmit the information in document A, encrypted,omitting graphics, to machine B”. Then a gestural sequence (morphemicsentence) to support this could be:

DEPRESS—the user presses on a displayed representation of A, indicatingthat A is to be selected for an upcoming operation

FLICK—the user flicks the device in the direction of device B,indicating that the operation is a transmission to B

FOLD—the user folds the top quarter of the device over the lowerthree-quarters, indicating that the transmission is to be encrypted.

TWIST—the user twists the device about its central axis, indicating thatthe data is to be “wrung out”, i.e. the graphics are to be omitted

SQUEEZE—the user squeezes the device, indicating that they areconfirming that the operation is to proceed.

Note that none of these gestures, in isolation, performs an act—the“ensemble” of temporally separated morphemes must be interpreted inorder to form the complete action.

DRAWING MODIFICATION SENTENCE

Consider a device which displays geometric shapes for user manipulation.Furthermore, one of the manipulations supported is to resize (orrescale) a shape. Furthermore, suppose that this resizing can be doneeither aliased (jaggy) or anti-aliased (edges are smoothed). Supposethat the user wishes to perform the command “Resize shape A by 120%,about the X axis only, using anti-aliasing”. Then a morphemic sentenceto support this could be:

DEPRESS—the user presses on a displayed representation of A, indicatingthat A is to be selected for an upcoming operation

STRETCH—the user stretches some portion of the device, indicating thatthe operation is to be a resize. When the user starts stretching, aportion of the status display displays “100”. The user continues tostretch until the status display reads “120”

RELIEF-MAP—the user “scores” the display by making a horizontal line ofdepression, indicating that the operation is to take place only aboutthe horizontal (X) axis.

DEPRESS—a circular thumb stroke is made in a different area of thedevice, indicating that anti-aliasing (smoothing the edges) is to beperformed.

DATABASE PRESENTATION SENTENCE

Consider a device which contains various personal information databases,such as a list of phone numbers, a list of addresses, and a calendar.Suppose that the user wishes the most appropriate of those databasesdisplayed. Then a gestural sequence to support this could be:

SPATIAL LOCATION—the user carries the device such that it is spatiallyproximate to either the telephone, the address book, or the refrigerator(where the family calendar is displayed), whichever is appropriate.

DEPRESS—the user touches the device to activate the operation. Thedevice now displays the personal information appropriate to thatlocation.

DATABASE RETRIEVAL SENTENCE

To extend the foregoing example of a database presentation sentence,consider two computers which contain calendar databases. If the userwishes to synchronize the calendars, a suitable gestural sequence tosupport this could be:

SQUEEZE—the user squeezes the device to activates its gesturalrecognition capabilities

ORBIT—the user orbits the device about the surface of the other device 3times, indicating a desire to only match data for the next 3 weeks.

MATCH—the user matches the edge of the device to the edge of thecalendar, indicating a desire to “match” contents between the twodevices.

PRINTER/COPIER CONTROL SENTENCE

Consider a device which can produce paper copies of documents. Supposethat the user wishes to tell such a device to produce a stapled, twosided copy of document A, enlarged to the next greater size. Then agestural sequence to support this could be:

DEPRESS—the user presses on a displayed representation of A, indicatingthat A is to be selected for an upcoming operation

RIP—the user introduces a spatial discontinuity into a portion of thedevice, indicating that the upcoming operation is to be a copy(“carrying away” some of the data).

PINCH—the user pinches the upper left corner of the device, indicatingthat the copies are to be stapled.

SQUEEZE—the user presses on the front and back of the device, indicatingthat the copy is to be two-sided.

STRETCH—the user stretches the device, indicating that the copy is to bean enlargement to the next greater size.

FACE—the device typically has its paper emitter on the bottom,preventing users from making copies accidentally. By facing the devicesuch that the emitter is on the side, the copying operation isinitiated.

LIGHT BASED CONTROL SENTENCE

Consider a device which can display documents. Suppose the user is usingthe document while seated on a train, and wishes the document to displayitself with a backlight when the train enters a tunnel, and wishes thedocument to display itself in a larger font when the train rumbles overrough tracks. Then a gestural sequence to support this could be

SQUEEZE—the user squeezes on the device, indicating that a loss of lightis to be compensated for by a backlight.

LIGHT—as the train enters a tunnel, the LIGHT gesture is made, and thedevice turns on the backlight.

WHACK—the user briskly raps the device against the palm of their hand,indicating that their desired preference for adjusting hard-to-readdocuments is to increase the font size.

VIBRATE—as the train goes over a bridge, the vibratory gesture issensed. because of the position of this VIBRATE morpheme (after thepreceding WHACK gesture) in this morphemic sentence, the device nowincreases the font size on the displayed text.

LIGHT—the user puts the device into his suitcase, making the LIGHTgesture. In this context (with no SQUEEZE before it), the LIGHT gesturecauses the device to power-off its display.

To better appreciate utility and construction of devices in accordancewith the present invention, several examples of devices are nowdescribed:

PORTABLE COMPUTER WITH SQUEEZE AND TILT CONTROL

A handheld portable computer 500 (e.g. a 3Com® PalmPilot®) capable ofbeing fitted with deformable, pressure sensitive edging 504 isschematically illustrated in FIGS. 35 and 36. The computer 500 supportsa name and address software application, providing a user viewablename-address entry field on display 503. In this embodiment, a user cansqueeze the deformable, pressure sensitive edging 504 (squeeze arrows507) of the computer 500. In response, the name and address softwareapplication causes the display 503 to animate by slowly incrementing(scrolling) through the name list from “A” towards “Z” entries. When theuser squeezes edging 504 again, the software application stops thescrolling animation. Scrolling functionality is further enhanced by theuse of a tilt sensor, which allows the computer's behavior to mimicconventional rotatable address books. If the computer 500 is tilted awayfrom the 45 degree angle at which someone might typically hold it, thescrolling rate is increased. In this application, the closer thecomputer 500 was tilted towards the user (as indicated by arrow 506 ofFIG. 36), the faster the scroll rate toward “Z”. However, if a usertilted the computer 500 back past the neutral 45 degree position (asindicated by arrow 506 in FIG. 36), the animation would move backwardswith a velocity related to the magnitude of tilt. In this way it waspossible for a user to search for items in a long list in a very naturalway, while only using one hand.

In an alternative mode schematically illustrated in FIG. 37, scrollingspeed can be completely controlled by pressure. The greater the squeezepressure (arrows 537), the faster the list scrolls. Release of theapplied pressure causes the scrolling to halt. In this alternative userinterface strategy, application tilt (as indicated by orthogonal tiltarrows 530 and 532) could be used to change the direction of thescrolling through the list, allowing a user to search portions of alarge two dimensional data set (schematically illustrated as a dataplane 520) that is not entirely visible either horizontally orvertically on display 503. By simply tilting the display 503 of computer500 as if it were a window through which the data plane 520 can beviewed, any particular portion of the data plane (for example, datasubset 524) can be viewed. As will appreciated, in both the foregoingmodes the speed of scrolling, the specific neutral tilt angle, andrequired pressures to initiate scrolling changes can be adjusted to fita particular user.

PORTABLE COMPUTER WITH HANDEDNESS DETECTION

Pressure sensors have been added to augment a conventional keyboard 551enabled user interface to a hand holdable Windows® CE class computer 550(i.e. a Cassio® Cassiopia®) schematically illustrated in FIGS. 38 and39. In this embodiment, user handedness was determined by using pressuresensors positioned on a right back-edge and a left back-edge of computer550. User studies have found that the difference in pressure between theright and left side gave a direct indication of handedness of a user. Asrespectively illustrated in FIGS. 38 and 39, handedness was used tojustify formatted text 554 to the left (FIG. 38) or right (FIG. 39),thereby allowing more space 555 on display 553 for an electronicannotation pen to be used to mark-up the text.

For both the embodiment of the invention illustrated by FIGS. 35-37, andthe foregoing embodiment illustrated in FIGS. 38 and 39, materialdeformation of a spongy, elastic, or otherwise deformable material mustbe measured. Although various techniques can be used to measure materialdeformation, including those based on imaging or fluid volumetricchanges, one particularly useful technique is based on the use ofpressure transducers. Commercially available sensors measure pressure(indicative of material deformation) by converting a pressure change toa change in electrical characteristics. For example, inexpensive sensorsthat change resistance in response to pressure can be obtained in avariety of shapes and sizes, including paper thin sensors, and easilybendable sensor strips. Sensors of this kind can be customized to anyparticular shape or form that a gestural Ul might require. The change inresistance is usually linearly related to pressure, with the sensorgenerally being placed in a potential divider network to model thepressure as a change in potential. For a practical circuit the resultingsignal needs to be amplified, buffered and translated such that thechange in value from minimum pressure to maximum pressure spans a usefulrange. The modified signal can now be fed into an analog to digitalconverter (ADC) to produce a digital representation of pressure. An8-bit ADC can typically be used for most applications, however, ifgreater sensitivity to pressure changes is required, a higher resolutionADC (e.g 16-bit ADC) can be used. As will be appreciated, the ADC couldbe memory mapped into the processor's address space as a peripheral, oralternatively supplied as a retrofitted pressure interface to existingcomputers capable of benefiting from this system. Since an RS232connection is an interface that is almost universally available onportable computers, one strategy is to have the parallel output of theADC converted into a serial RS232 frame using a serializer such as aUART and then level-shift and buffer the signal as specified by theRS232 standard. At the computer end of the serial interface, anotherlevel-shifter and UART, the output of which is readable by theprocessor, performs the serial-to-parallel conversion.

As described with reference to FIG. 40, in realizing a working system amicrocontroller 564 (a Signetics 87c752) with an ADC built in to combinemany of the I/O tasks in a single chip can be used in combination withlevel shifter 566 (a MAX3223). This approach has the advantage thatintelligent processing of the input signal is possible with software.Tilt measurement was provided by a tilt sensor 567 connected to abuffer/amp 562 to supply an analog signal to microcontroller 564. Thepressure measurement can also be encoded within a protocol across theserial link.

This particular microcontroller 564 has five ADC inputs, but by makinguse of eight digital control lines, it is possible to use only one ADCinput and one buffer amplifier 561, to measure up to eight pressurepoints with pressure sensors 565. This is achieved by using the controllines to select only one sensor at a time and take a reading for eachusing a single input to the ADC. After eight sensors have been selected,eight readings are acquired in memory. The design is practical becausethe microcontroller can take measurements and make analog to digitalconversions at a rate far higher than is needed to communicate with ahost computer 569.

For scrolling or handedness based software applications, 16 levels weredetermined to be adequate for representing a pressure measurement. Inorder to have a high data throughput to the host computer 569, eachmeasurement was encoded in one byte of an RS232 frame such that the fourlowest bits were the pressure representation and the highest four bitswere the sensor ID. Each frame of the RS232 data was thereforecompletely self contained. Of course any protocol that limits itself toa specific number of devices in its address space will eventually havethe problem that someday applications might be designed that need toreference far more devices than are supported. The solution used in thisprotocol is to reserve the sensor ID number 15 as a special value thatcan extend the semantics of the encoding to include an arbitrary numberof bytes representing a sensor or value. For the described softwareapplications, the commonly used RS232 frame format (1 start, 8 data noparity, 1 stop bit at a baud rate of 9600) was chosen.

In operation, the host computer 569 determined handedness by utilizinginformation about the current pressure exerted on two subregions—one, onthe back of the device, occupying roughly the left half, and the other,also on the back of the device, occupying the right half. The currentpressure values were converted from analog to digital form, such that adigital value of 0 (zero) represents no pressure, and a digital valueof, for example, 15 represents maximum pressure. The detection circuitrythen proceeds:

if (left_sensor is high AND right_sensor is high) then conclude that theuser is gripping the device with both hands

else if (left_sensor is high AND right_sensor is low) then conclude thatthe user is gripping the device with the left hand only

else if (left_sensor is low AND right_sensor is high) then conclude theuser is gripping the device with the right hand only

else if (left_sensor is low AND right_sensor is low) then conclude thatthe user is gripping the device with neither hand

Also, to optimize communication, pressure values are only sent when theychange. In order to account for jitter and error in the pressuresensors, a sensor is only considered ‘high’ if its value is higher thansome minimum threshold (e.g. ‘2’ on the 0 to 15 range).

SCANNER/PRINTER/COPIER SUPPORTING EDGE DEFORMABLE DISPLAY

A scanner/printer/copier device 570 is schematically illustrated in FIG.41. As seen in FIG. 41 (and in more detail in FIG. 42) the device 570supports a display 574 having a deformable edge 572, with thecombination roughly shaped like a piece of paper. In operation, a usercan place a written document in the device 570 for scanning. Anelectronic version of the scanned document is displayed (i.e. as text575 in FIG. 42) on the display 574. By outwardly pulling the deformableedge 572 as indicated by arrow 577, a user can instruct the device 570to resize the document before printing or copying. Pinching opposingsides of the deformable edge 572 (arrows 578) can further instruct thedevice 570 to print double sided copies. As will be appreciated, variousother morphemes can be used to provide a simple interface forinteracting with device 570.

TILEABLE AND STACKABLE PORTABLE DISPLAYS

Multiple autonomous display tiles having an onboard display controllerand at least one surface consisting substantially of a conventionaldisplay are particularly useful for the practice of various aspects ofthe present invention. Such tiles can be interconnected in response tovarious morphemes such as TOUCH, FLICK, RELATIVELY ALIGN, or WHACK, oreven can be operated without substantial user mediated morphemic inputin certain situations.

Advantageously, tile positioning can be used as an interface specifierin and of itself. For example, each display tile can contain a videosegment in independent memory. Shuffling or reorganizing the tiles canallow users to physically manipulate the sequence of video segments toaffect a physically manipulatable video editing system. Using the cardanalogy, tiles can be used to re-sequence documents, pages within adocument, audio annotations, voice mail, or other temporal mediacontained within the tiles. The resultant sequence can then be played asa whole by using the tiled structure as a unit.

For purposes of the present invention, display tile array configurationscan be categorized as follows:

Close-packed display tile array 600 (FIG. 43) in which tiles 602 arearranged to span a surface 610 in closely abutting, but not overlappingrelationship, such that the continuous display area is maximized (i.e.seamlessly). As will be appreciated, the tiles could themselves form afreestanding continuous surface, or they could be disposed on a table orother suitable support. Each of the tiles 602 supports a display 604sized to substantially cover a front surface of each tile 602. Incertain embodiments, a back surface of each tile 602 can also support adisplay. Advantageously, this would allow for creation of free standingdisplays with images visible on the front and back. The surface 610 canbe a plane, a sphere, or any arbitrary shape that permits tiling.

Loose-packed display tile array 620 (each tile 622 having a display 624as seen in FIG. 44) in which tiles 622 are slotted into a latticepattern (indicated by dotted lines 625). Each tile can be considered tobe situated in a regular bounded lattice slot (namely, the lattice slotthat contains a defined center of each of the tiles) within the latticewhere the dimensions of the bounded slot are no more than a few timesthe maximum dimension of the tile, and there is no more than one tile inany slot. Within the lattice regions any of the tiles can be positionedarbitratrily and still retain the same inter-relationship with thegroup. It is still possible for tiles to touch one another at theboundaries of two or more lattice slots, however, this is not arequirement.

Free-format display tiles 630 (each tile 632 having a display 634 asseen in FIG. 45) are similar to loose-packed display tiles 620 where thesize and shape of the lattice slots may vary freely (e.g. the dimensionof a lattice slot may be many times the maximum dimension of any of theparticipating tiles). The only constraint on the arrangement is thatthere must be no ambiguous relationships about the relative connectivityof the each of the tiles 632. That is to say, a tile that is to displaythe next piece of information to one side of another tile must be uniqueand not be confused with the task of another tile in the tiling lattice.

3D display tiles (packable display tiles) are created by extending thethree foregoing display tile categories. However for close packed tilingof display tiles, packed devices in the center of a 3D structure wouldnot be available as the user interface. This may not matter as thesurface of the 3D shape will expose a area that will have uniqueaffordances for some classes of application. For instance, cubic tilespacked into the shape of a large cube can display, using the six facesof the larger cube, the various projections that could be rendered byviewing a 3D CAD drawing from each degree of freedom.

As will be appreciated, display tiles need not be recti-linear but canbe hexagonal, circular, or of arbitrary shape and size. The size oftiles need not be constant for all tiles within a larger structure.Tiles need not be aligned but may require proximity to indicateadjoining points or edges.

Tiles need not be in physical contact to define they are joined in agroup activity but instead this may be a programmed function. Theconnectivity can also be discovered through a radio network from acoordinating server or from a distributed algorithm that draws in asmany computers as necessary for a task, using the wireless network tonegotiate the resources it needs. The tiled computers may also beconnected by wired networking systems, although in cases where thetopology might need to rapidly change, this is not as desirable as awireless system. An example of a wired networking system of this type isa system that uses the internet to include many computers in a singletask, one in which each computer knows its relative position even thoughthey may be separated by rooms or in the extreme cases buildings, citiesor countries. However, in general the most useful case of display tilingis when the tiles are proximate enough that they can all be viewed byone person to create a display medium whereby the viewing experience isan enhancement over using a single display. Accordingly, tiles canoperate as either a single larger contiguous structure or they canretain individual properties and independent functions or a combinationof both. (e.g., jumbo-tron like function, 12×12 small displays as usedin a TV studio, or picture in-picture features found in commercial TVsor editing suites).

Depending on the type of packing, various schemes can be used to allowfor permanent, intermittent, or even one-time communication betweendisplay tiles. For example, close packed tiling can make use of wiredconnectivity between computers or could use a variety of wireless oroptical communication technologies. In the case of wired connectivity,edge mounted conventional plug and socket connectors may be used tocreate a rigid tiled array. Plug and socket systems lend themselves toparallel connections for bulk and high-speed data transfer. They alsoprovide a convenient method of power distribution, which can allow forone of the tiles providing a power source that supports the rest of thearray. A plug and socket connection between tiled computers 692 in atiled computer array 690 (with both data and power transfer shown) isschematically illustrated by lines 695 in FIG. 48.

The exact design and locations of electrical connectors depends on theintended use, and subsequently, the shape of tile components. Forapplications where large seams between displays are acceptable, simplerigid connectors could be attached to the center of each edge, providingconnectivity to all the surrounding tiles. Other applications mightrequire a more complex design. For example, applications which requirehigh-quality information display on a seamless array surface (i.e. tiledblueprints), might employ spring-loaded contacts on the four edgeconnectors. The spring mechanism allows all array connections to be madebelow the display surface, while tiles can still be inserted and removedfrom the interior of the array. Removal of tiles could be triggered bysome host signal which actuates a release of the spring and pops thetile out of the array.

Serial connectivity can also be used in a close packed arrangement suchas that illustrated in FIG. 43. It has the advantage that fewerconnections have to be made and in practice it might be more reliable.However the net bandwidth between tiles will be less than a parallelsystem. Serial communication lends itself to optical and wirelesssystems thus removing the need for any physical connections. For opticaltechnologies alignment of the transmitter and receiver is stillimportant although the clever use of light pipes and lens capturetechniques can introduce more flexibility. Wireless systems can use manydifferent bands of the EM spectrum (kHz, MHz, GHz), utilizing a varietyof modulation techniques (amplitude modulated, frequency modulated, orthose based on code division multiple access) and operate at a range oftransmitter powers. There is no longer a need for direct alignment ifthe system is designed with suitable communication tolerances. Thetransmitter range plays a crucial part in the design. If there is onlyenough power in an emitted signal to be picked up within a fewmillimeters of a tile edge, then the signals will be isolated, thetopology will be defined by the physical connectivity and the complexityof designing the system to avoid interference from neighboring signalsources will be minimized. However, an alternative design is to use morepowerful radios. In this case all tiles will be able to contact allother tiles and inter-tile connectivity needs to be defined by anotherparameter. Signal strength can be used or, more deliberately,information that relates the ID of a tile to a spatial map (perhaps heldin one master tile) describing the position of all tiles in the tilearray. In this system it is also necessary to minimize inter-tileinterference. For digital packet-data systems that operate at the samefrequency, carrier sense multiple access (CSMA-CD or CSMA-CA) systemsare well known techniques to solve this problem. Other solutions involvetiles using different frequencies, with frequencies reused depending onthe power of the transmitters. This is the technique used by traditionalcellular telephones. Yet another approach is to use Code DivisionMultiple Access (CDMA) that relies on the overlaying of signals in thesame region of the EM spectrum, a technique known as spread spectrummodulation.

For loose packed display tiles, the wireless techniques described abovein connection with close packed tiling generally become essential to theimplementation. However, a special case of loose packing exists in whicheach edge of a tile does make contact with every other surrounding tile,except that it may only be a single point of contact and not at aaccurately defined place. A wired version of this system can be built inwhich the entire edge of each tile is a serial connection including oneof the two vertices that define the edge. Communication in eachdirection can be achieved by a variety of commercially availabletechniques, including use of a one-wire interface (plus a ground) forbidirection communication between tags and a reader. Note that theground connection for a tile arrangement can be derived by sharing acommon ground connection through the surface the tiles are laid out on.For example, the surface could be made of a metal sheet. The system maybe further enhanced by ensuring that the edge contact is made of amagnetic material and the vertices have a magnet embedded in the end.Such an arrangement ensures that there will be a good electrical contactbetween the transmitter and the receiver.

The receiver can also derive power from an electrically transmittedsignal by bridge-rectifying it and storing the collected charge in acapacitor for use by its own electronics. Thus, power distribution canalso be included in a one-wire interface. In this way flexibleconnectivity can be achieved to support the rapid and convenientrearrangement of tiles in which only the minimum amount of care needs tobe taken in setting up connectivity.

Loosely packed tile displays such as illustrated in FIG. 45 may requirethat the display surfaces use a best effort algorithm to present aunified display with all the sections of the displayed image bearing thecorrect spatial arrangement to each other even though, in the case ofrectangular tiles, they might not be registered vertically orhorizontally and have an offset angle relative to each other. In orderto implement the desired tile display algorithm, not only is therelative arrangement of tiles important but the exact offset (distanceand angle) from each other is also important. There are several methodsthat are suitable for automatically determining offsets between looselypacked tiles. For example, as illustrated with respect to FIG. 46(showing communicating tiles 652 and 654, with respective displays 651and 653), optical encoding 660 along an edge can be used to identifytile orientation. It is possible to use a binary coded optical patternalong an edge which is regular and encodes the distance from a vertex atany point. A tile that abuts, or is relatively aligned, can read thispattern using optical sensors 658 and 659 and determine the displayoffset in the direction of the edge. Alternatively, as illustrated byFIG. 47 (showing communicating tiles 672 and 674), it is possible to useradio based techniques that rely on signal strength triangulation. Eachvertex 675 or 676 of a tile 672 can contain a wireless transmitter and areceiver. If these vertices transmit a short characteristic wirelesssignal at well known times, a nearby tile 674 can use the receiverssituated at its own vertices 680, 681, 682, and 683 to triangulate theposition of each transmitting vertex relative to them by measuring therelative delay by which the signal was received. After two of thevertices of the original transmitting tile 672 have signaled, theadjoining tile 674 can determine its exact position and orientationwithin their local region of a tile array. The transmitting andreceiving tiles can now swap roles, with the result that both tiles willknow their relative position. This process can be extended across thetile array.

Free format tiling differs from loose-packed tiling in that there are noproximity, or regular format, constraints. To demonstrate how a freeformat tiling system might work, the following example is described.Imagine a number of laptop computers each fitted with a GlobalPositioning System (GPS) and a radio modem. Each laptop can determineits position (with acceptable error) and communicate it to all the otherlaptops by contacting them through the radio modem. After some period oftime the laptops will all know their relative location and absolutelocation. If any computers changes its location, it can update the localneighbors to ensure there is enough understanding in the array toconsider the computers are in a known tiling configuration even thoughthey are not close to each other and in fact they may be in differentgeographic regions. An application that might use this free formattiling system is one that wishes to ensure information is beingcorrectly sent in a uniform and dispersed fashion across a very largearea. For example, the information that each laptop computer receivescould be instructions to release an amount of a pesticide used tocontrol an insect that does damage to commercial crops. If the pesticideis released in too high a concentration in any locality, it may behazardous to human health. The tiled approach allows the computers toroam (e.g. in the back of a truck), displaying information as to thetype and concentration of pesticide that is to be released given theircurrent relative proximity.

Various conventional algorithms can be used to support distribution ofinformation between autonomous tiled displays. These algorithms assume asystem by which there is a master controller generating data to bedisplayed. There is also a large array of tiled computers that thesystem will use to display visual data and/or process information. Eachcomputer in the tiled array contains a unique ID. It is the job of themaster to split the data into pieces that each computer can display andfor this information to be packaged along with the ID of the targetcomputer. The algorithms below describe how the information travels fromthe master to a destination display tile:

Daisy Chain Routing

Display tiles are arranged to have a logical connectivity with eachother so that each one only transmits information to the next in apredefined line. The computers are said to be daisy chained to each. Anyinformation sent to the start of the chain contains an ID and the firstcomputer in the chain compares it to its own ID. If it matches, it actson the data. If not, it sends the data on to the next computer in thechain until it finds its destination.

N-ary Routing

In N-ary routing the path to the destination is contained in the ID ofthe device. A simple routing example is schematically illustrated withrefrence to the direction arrows 695 in FIG. 48, which shows quaternaryrouting in physically connected tileable displays. In quaternaryrouting, an array is conceptually arranged as a quaternary tree witheach node having an input and three outputs. In this system each pair ofbits of the ID contains a routing command. A 0 indicates send the packetto the first output, a 1 indicates the second output, a 2 indicates thethird output, and 3 indicates no further transmission. There is also acount that gets decremented by each node to tell successive nodes thebit number that is currently being considered and when the packet hasreached its destination. In this way packets are forwarded from node tonode with a simple choice at each stage until they reach their targetdisplay. As will be appreciated, it is possible to design N-ary systemswith more than three outputs (a power of two is usually convenient forimplementation (e.g. 4, 8, 16 . . . )).

Flooding

Flooding has no predefined routing structure. The computer that takesthe first packet from the master checks to see if it has the correct ID.If not, the packet is sent out on all links to which it has not yet sentor received that packet. The result is a flood of copies of the packetacross the array eventually reaching its destination. The packets mustalso have a maximum hop count to ensure they eventually are removed fromthe system. The disadvantage of this approach is that many more tilesare burdened with the processing of unwanted data than in the previoustwo schemes, which may impact the overall efficiency of the system.

Hot Potato

The Hot Potato algorithm is similar to the Flooding algorithm, except aretransmitted packet is sent out on only one output that is eitherchosen randomly or is the least busy. The process stops when the packetreaches the correct tile. The time that a packet takes to reach itsdestination is not deterministic.

As those skilled in the art will appreciate, other variousmodifications, extensions, and changes to the foregoing disclosedembodiments of the present invention are contemplated to be within thescope and spirit of the invention as defined in the following claims.

What is claimed is:
 1. A reconfigurable intelligent system comprising aplurality of 3-dimensionally tilable display devices for transferringdata, the reconfigurable system comprising: a first device having aprocessor, a first spatial position sensor, and a first wirelesscommunication module for transferring data; a second device having aprocessor, a second spatial position sensor, and a second wirelesscommunication module for transferring data; and a third device having aprocessor, a third spatial position sensor, and a third wirelesscommunication module for transferring data; and the first device isconnected in one or more of an intermittent or continuous wirelesscommunication with the second device and the third device to pass datafrom the first, second, and third spatial position sensors of therespective first device, second device, and third device; and thereconfigurable system is reconfigurable based on the data from thefirst, second, and third spatial position sensors.
 2. The reconfigurablesystem of claim 1, wherein the first device maintains information aboutspatial position of the second device and the third device, andtransfers differing data to the second device and the third device basedon that maintained spatial position information.
 3. The reconfigurablesystem of claim 1, wherein the first, second and third wirelesscommunication modules comprise one or more of a radio link and anoptical link.
 4. The reconfigurable system of claim 1, wherein thefirst, second and third devices further have a respective first, second,and third deformable piece, with intercommunication between at least twoof the devices being triggered by a touching action between devices. 5.The reconfigurable system of claim 1, each first, second and thirddevice respectively further comprising a first, second, and thirddeformable piece, with intercommunication between at least two of thedevices being triggered by a stacking action between devices.
 6. Thereconfigurable system of claim 1, each first, second and third devicerespectively further comprising a first, second, and third deformablepiece, with intercommunication between at least two of the devices beingtriggered by a tiling action between devices.
 7. The reconfigurablesystem of claim 1, each first, second and third device respectivelyfurther comprising a first, second, and third deformable piece, withintercommunication between at least two of the devices being triggeredby a wrapping action between devices.
 8. A reconfigurable systemcomprising a plurality of tileable devices for transferring data, thereconfigurable system comprising: a first device having a processor anda first wireless communication module for transferring data; a seconddevice having a processor and a second wireless communication module fortransferring data; and a third device having a processor and a thirdwireless communication module for transferring data; wherein the firstdevice is in wireless communication with the second device, the wirelesscommunication being one or more of an intermittent or continuouscommunication, with the first device maintaining information aboutspatial position of the second device, and the second device maintaininginformation about spatial position of the third device, and with thefirst device implementing a first data set, a second data set beingtransferred from the first device to the second device forimplementation, and a third data set being transferred from the seconddevice to the third device for implementation.
 9. The reconfigurablesystem of claim 8, wherein the first device maintains information aboutspatial position of the second device and the third device, andtransfers differing data to the second device and the third device basedon that maintained spatial position information.
 10. The reconfigurablesystem of claim 8, wherein the first, second and third wirelesscommunication modules respectively support infrared links.
 11. Thereconfigurable system of claim 8, wherein the first, second and thirdwireless communication modules respectively support radio links.
 12. Thereconfigurable system of claim 8, each first, second and third devicerespectively further comprising a first, second, and third deformablepiece, with intercommunication between at least two of the devices beingtriggered by a touching action between devices.
 13. The reconfigurablesystem of claim 8, each first, second and third device respectivelyfurther comprising a first, second, and third deformable piece, withintercommunication between at least two of the devices being triggeredby a stacking action between devices.
 14. The reconfigurable system ofclaim 8, each first, second and third device respectively furthercomprising a first, second, and third deformable piece, withintercommunication between at least two of the devices being triggeredby a tiling action between devices.
 15. The reconfigurable system ofclaim 8, each first, second and third device respectively furthercomprising a first, second, and third deformable piece, withintercommunication between at least two of the devices being triggeredby a wrapping action between devices.
 16. A method for transferringinformation between a first display device and a second display device,the method comprising the steps of: connecting the first device and thesecond device in one or more of a intermittent or continuous wirelesscommunication; flicking the first device to provide a morpheme input tothe device; and triggering transfer of information from the first deviceto the second device in response to the flicking action.
 17. A methodfor transferring information between a first display device and a seconddisplay device, the method comprising the steps of: connecting the firstdevice and the second device in one or more of a intermittent orcontinuous wireless communication; facing the first device relative to asecond device to provide a morpheme input to the device; and triggeringtransfer of information from the first device to the second device inresponse to the facing action.